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Abstract 

In this paper I describe how miscommunication 
problems are dealt with in the spoken language 
system DIALOGOS. The dialogue module of the 
system exploits dialogic expectations in a twofold 
way: to model what future user utterance might 
be about (predictions), and to account how the 
user's next utterance may be related to previ- 
ous ones in the ongoing interaction (pragmatic- 
based expectations). The analysis starts from the 
hypothesis that the occurrence of miscommuni- 
cation is concomitant with two pragmatic phe- 
nomena: the deviation of the user from the ex- 
pected behaviour and the generation of a conver- 
sational implicature. A preliminary evaluation of 
a large amount of interactions between subjects 
and DIALOGOS shows that the system perfor- 
mance is enhanced by the uses of both predictions 
and pragmatic-based expectations. 



The problem 

During the last few years it has been emerging that 
the success of spoken language systems is greatly im- 
proved by the contextual reasoning of dialogue mod- 
ules. This tenet has spread through both the speech and 
the dialogue communities. Dialogue systems devoted to 
spoken language applications are able to detect partial 
communication breakdowns by other system modules, 
and that increases the robustness of human- machine in- 
teractions by speech. 

During oral interactions with computers, communica- 
tion problems often arise after the occurrence of errors 
during the recognition phase. Sometimes these errors 
cannot be solved by the semantic module: the utter- 
ances containing them are interpreted by the seman- 
tic analyser, but with an information content different 
from the speaker's intended meaning. Detecting such 
miscommunications and repairing them through initial- 
ization of appropriate repair subdialogues is essential 
for the interaction to be successful. 

Most of the research in this area has been devoted to 
providing the recognition and understanding modules 
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with information generated on the basis of the dialogue 
context. They predict what the next user's utterance 
will probably be about: throughout this paper I will 
refer to them with the name "predictions". Sometimes 
they are passed down to the acoustic recognition level 
in order to decrease the huge number of lexical choices, 
sometimes they are used to help in deciding on multi- 
ple semantic interpretations. More often they are used 
during the contextual interpretation phase to accept or 
reject parser output. 

Although predictions have proved useful, they only 
grasp one side of the miscommunication problem. Ac- 
tually they are a means for reducing recognition errors, 
and their use allows the avoidance of one of the po- 
tential sources of miscommunication. However during 
spoken human-computer interactions, the detection of 
miscommunications may be outside the capabilities of 
the dialogue system, even though it uses predictions; 
on the contrary, the user is usually able to detect any 
speech errors. For example, in travel inquiry applica- 
tions words belonging to the same class, such as proper 
names of place, may be highly confusable. When the 
dialogue prediction says that next user's utterance is 
likely to be about a departure place, this does not ex- 
clude that the recognizer substitutes the actually ut- 
tered name with a phonetically similar one. Only the 
user is able to detect such kinds of errors. In this sit- 
uation the dialogue system should identify the user's 
detection of miscommunication and provide appropri- 
ate repairs. 

In this paper I will argue that the dialogue mod- 
ule ability to detect user-initiated repairs is improved if 
the system is able to capture the pragmatic phenomena 
that accompany user's detection of miscommunication. 
The paper offers an analysis of the pragmatic phenom- 
ena that occur when users detect miscommunication 
during task-oriented human-machine spoken dialogues. 
The account exploits both predictions and another no- 
tion of expectation that comes from the cognitive-based 
research area. This notion refers to conversants' be- 
liefs about the relation of future utterances with pre- 
vious ones in a dialogue. A computational interpre- 
tation of this notion has been d one in the model pro- 
posed by (McRoy and Hirst 1995), where it is suggested 



that speech community predictions and cognitive based 
expectations are complementary notions. This paper 
claims that accounting for these two notions is useful 
for detecting and solving actual breakdowns in user- 
system communication. The working hypothesis is that 
in task-oriented dialogues miscommunication often gen- 
erates conversational implicatures. I will show how they 
are dealt with by the dialogue module of DIALOGOS, 
an information inquiry spoken language system imple- 
mented by Cselt speech recognition and understanding 
group. By using the telephone, the system may be used 
to access the data base of the Italian public railway com- 
pany. I will report dialogue examples and experimental 
data that show the effectiveness of the proposed analy- 
sis in task-oriented human-machine spoken dialogues. 

The working hypothesis 

The conceptual background of this approa ch is inspire d 
by the Gricean principles of conversation (Gricc 1967). 
The user modelling of the dialogue module of DIAL- 
OGOS is based on the assumption that both the sys- 
tem and the user are active agents of the communica- 
tive process; in particular, it is assumed that both of 
them observe the Cooperation Principle (CP) in or- 
der to achieve the general goal of their linguistic inter- 
change, i.e. to access a database to get the information 
that the user needs. 

The system predictions are modelled on the basis of 
the CP. At each stage of the dialogue with the user 
the system expects that user's reaction conforms with 
three of the original Gricean conversational maxims, 
re-interpreted i n the contex t of human-machine com- 
munication in ( Eraser 1992| ). For example, in this ap- 
plication domain, throughout the dialogue the system 
expects that: 

• each user turn is not over or under-informative, 

• each user turn communicates user's "true"' needs (i.e. 
to receive train timetables from place X to place Y, 
if the two places have been confirmed), 

• is pertinent to the focus in hand. 

However, as remarked earlier, in oral human-machine 
dialogue the communication process may be disturbed 
by several factors, the most usual of ones are recog- 
nition errors. The user's detection of breakdowns and 
errors of the system has precise empirical consequences: 
these concern both user's behaviour and her cognitive 
demand on the continuation of the interaction. In par- 
ticular, these empirical consequences may be summa- 
rized as follows: 

• the user utterance does not match dialogue predic- 
tions; 

• the user asks the system to come back to the inter- 
pretation context where miscommunication occurred; 

• both the user and the system should engage in a clar- 
ification subdialogue before continuing their interac- 
tion. 



Since dialogue predictions are generated by taking 
into account the respect of the above listed maxims, and 
given that the system assumes that the user continues 
to respect the CP, each deviation of the user's behaviour 
from dialogue predictions is ideally interpreted as a sig- 
nal of the potential occurrence of miscommunication. 
By co-operating in the achievement of the goal of the 
conversation, the user intentionally violates the maxims 
when she detects that a misunderstanding occurred in 
previous turns. Accordingly to the conceptual back- 
ground here adopted, the intentional violation of the 
conversational maxims gives place to a conversational 
implicature. 

The working hypothesis is that the occurrence of 
a miscommunication goes along with two pragmatic 
counterparts: the first one is the deviation of the user's 
behaviour from the dialogue predictions, the other one 
is the generation of a conversational implicature. the 
turn For being "conversational" an implicature has to 
be inferable on the basis of the contextual knowledge. I 
suggest that in order to recognize conversational impli- 
catures, the dialogue system should embody a model of 
user expectations that gives insights on how to relate 
user correction turns to the previous (normal) turns in 
an ongoing dialogue. 

Examples of miscommunication 
Non-understanding 

The kind of miscommunications that occur during 
oral interactions of subjects with spoken language sys- 
tems is usually caused by an actual breakdown in the 
flow of the dialogue. The most usual miscommunica- 
tions may be labelled as "misunderstanding" and "non- 
understanding". The latter are usually less problem- 
atic. 



Tl S: Automatic Railway Information System. Please 
speak after the tone. Please give your point of 
departure and your destination. 

T2 U: Then ... What time .. I mean from [NOISE] eight 

< Recognition and understanding failure> 

T3 S: Sorry, I did not understand. Please give your 

point of departure and your destination. 

T4 U: I want to leave from Milano in the evening. 

I 'd like to know departure [NOISE] from 

Milano to Roma. 

< confirmation— NO, departure-city— MILANO, 
arnval-city=ROMA, departure-time=EVENING> 

T5 S: Do you want to travel from Milano to Roma 

leaving in the evening? 

T6 U: Exactly. Around eight p.m. 
< confirmation= YES hour=EIGHT> 

T7 S: Intercity 243 leaves from Milano Centrale at 

20 past 8 p.m.; it arrive at Roma Termini 6 a.m. 



Figure 1: Example of non- understanding 
In the dialogue reported in Figure non- 



understanding occurred in the turn T2: due to environ- 
ment noise, the fragmentary utterance was not under- 
stood by the recognizer and no semantic representation 
was sent to the dialogue module by the parser. This 
symptom of actual breakdown causes the system to trig- 
ger an informative speech act of non-understanding and 
a requestive speech act for obtaining departure and ar- 
rival place names (T3). 

In non-understanding the system completely fails to 
interpret a user's utterance; that may happen either 
because a speech recognition error occurred or because 
the linguistic processor was not able to interpret it, 
for example if the semantic content of the sentence 
was out of its semantic coverage. Non-understanding 
is usually recognized by the dialogue system as soon 
as it happens. Within speech understanding systems 
non-und erstanding usu ally gives place to second-turn 
repairs (ISchegloff 199^ ). For example, in the DIALO- 
GOS system the dialogue module realizes that a non- 
understanding has occurred because it fails to receive 
any semantic interpretation for the user's sentence ut- 
tered immediately before. In this case it enters a re- 
pair action by informing the user that her utterance 
was not understood and asking her to rephrase the sen- 
tence. At present the dialogue module does not know 
if the non-understanding was caused by a failure of the 
speech recognizer (because of out-of- vocabulary words, 
mispronouncing, unhearings, and so on) or by a fail- 
ure of the parsing module. As a consequence, the 
user is addressed with a generic information of non- 
understanding. 

Misunderstanding 

Some kinds of failures in recognition and understanding 
cause actual human-machine misunderstanding. Let us 
consider the dialogue in Figure ||. The user utterance 
T2 has been recognized as "Arona" instead of "Roma" 
(in Italian they may be confusable). This causes a third- 
turn repair. In the turn T4 the user, instead of giving 
the departure place name, realizes that the sentence ut- 
tered in the turn T2 was not correctly recognized and 
she repeats the arrival city. In producing T3, the sys- 
tem generated a prediction about the departure city, 
but user's turn T4 causes it to recover. This is done 
by reinterpreting T4 within the interpretation context 
initially generated for T2. Moreover the expectations 
generated in interpreting T4 are different from the ones 
generated for T2. In T4 the system realizes that a mis- 
understanding happened: it discards predictions about 
obtaining the departure city by refocusing the inter- 
pretation context on the possible confirmation of the 
arrival city. 

Identifying misunderstanding 

The example reported in Figure |^ is an actual piece 
of dialogue (translated into English) between a sub- 
ject and the DIALOGOS system. Let us now describe 
how the system deals with the misunderstanding. In 



Tl S: Automatic Railway Information System. Please 
speak after the tone. Please give your point of 
departure and your destination. 
T2 U: I want to go to Roma. 

< *arrival-city= ARONA > 

T3 S: To Arona. What is your point of departure? 
T4 U: I said that I was going to Roma. 

< arrival-city =ROM A > 

T5 S: Arc you going to Roma? 



Figure 2: Example of misunderstanding due to substi- 
tution 



the turn T3 the dialogue system generates a so-called 
"confirmation plus initiative" speech act. This means 
that in a single dialogue turn the system tries to get 
both a confirmation of what it understood from the pre- 
vious user turn, and other piece of information it needs 
to access the database. The selection of this speech act 
generates a large number of expectations. The most 
interesting for the present discussion are listed below. 
For brevity, in what follows I have considered only the 
predictions of statements with more informational con- 
tent than simple confirmation and denial by the adverbs 
"yes" and "no". 

1. A statement about the name of the departure city; 

2. A statement about the name of the departure city 
and other required parameters, such as the date and 
the time of departure; 

3. An explicit confirmation of the arrival, and the de- 
parture city; 

4. An explicit denial of the understood arrival city and 
a request for another arrival city; 

5. An explicit denial of the understood arrival city; 

6. A statement including a new arrival city plus the de- 
parture city; 

7. A statement including a new arrival city; 

As we can observe, the expectations from (1) to (7) 
may be related to predictable lexical counterparts. In 
other words, they may be sent to the recognition level 
in the form of word class predictions, in order to be 
used for constraining the search space of the speech 
recognizer. 

In addition, we can observe that at the dialogue level 
the realization of expectations (6) and (7) implies some- 
thing more than the prediction of a lexical class. The 
linguistic realizations of (6) and (7) have the form of af- 
firmative statements and the illocutionary force of dis- 
confirmations. In both cases a conversational implica- 
ture is generated. The illocutionary goal of statements 
realizing (6) and (7) imply a precise cognitive demand 
by the user. Let us again consider Figure ||. T4 realizes 
dialogue expectation (7): the user asks the system to 
reinterpret T4 in the context of T2. Notice also that 



the user's beliefs about the continuation of the interac- 
tion are sHghtly different for (6) and (7): in the case of 
(6) the focus of attention of both user's and system are 
still on the same speech act, i.e. the disconflrmation 
of the arrival city and the proposal of a departure city. 
On the contrary, in (7) the user expects that before re- 
questing a new parameter the system will be able to 
correct its previous misunderstanding. In other words, 
she demands a focus restriction in order to correct the 
misunderstanding. An appropriate system answer in 
situations like (6) and (7) requires that the system be 
able to grasp the illocutionary force of user statements 
in order to redirect the dialogue strategy. 

Dealing with misunderstanding 

Let us consider in Figure ^ another excerpt from an 
interaction between DIALOGOS and a subject. 



Tl S: Automatic Railway Information System. Please 
speak after the tone. Please give your point of 
departure and your destination. 
T2 U: From Milano to Roma. 

< departure- city = MILANO *arrival-city=ARONA > 
T3 S: Do you want to travel from Milano to Arona? 
T4 U: From Milano to Roma. 
< departure- city— Milano arrival-city— ROMA > 
T5 S: Do you want to arrive in Roma? 



fact that T4 is now being reinterpreted in the context 
of T2 causes a refocusing of the dialogue context. In the 
present DIALOGOS version the refocusing mechanism 
is allowed through the maintenance of both a linguistic 
history and a context hierarchy (further details of this 
are reported in the section that describes the dialogue 
system) . 

All the examples discussed above concern third-turn 
repairs; however, repairs from misunderstanding may 
occur also in subsequent turns, as the example in Figure 
^ shows. 



Tl S: Automatic Railway Information System. Please 

speak after the tone. Tell me from where you 

want to leave and where you want to go. 

T2 U:... leaving from Pisa at eight [NOISE] this 

morning [NOISE] Firen- 

< *departure-city=PISA-AEROPORTO 
departure-time=MORNING date=TODAY > 

T3 S: From Pisa Aeroporto. Where are you going to? 

T4 U: To Firenze. 

<arrival-city=FIRENZE> 

T5 S: From Pisa Aeroporto to Firenze? 

T6 U: From Pisa ... Stazione Centrale to Firenze 

< departure-city =PIS A dep-station=CENTRALE 
arrival-city=FIRENZE> 

T7 S: From Pisa Centrale? 
T8 U: Yes 

< confirmation=YES > 



Figure 3: Example of repair with repetition 

In the turn T3, DIALOGOS prompts the user for 
confirmation with a Yes/No question. At this point 
the system linguistic predictions concern a confirma- 
tion or a negation speech act by the user. In T4 the 
user produces an affirmative sentence which has the il- 
locutionary force of a disconflrmation. By analysing 
a large number of interactions between users and the 
DIALOGOS system, we actually realized that in this 
application context Italian speakers very often start a 
repair without an explicit negation word such as "No": 
on the contrary, they repeat their previous utterances 
louder. Notice t hat in Italian rep etitions mostly work 
for confirmation ( Bazzanella 1993| ), but in that case the 
intonational contour is unstressed. 

As proposed above, in order to recognize the dialogic 
repetition of the turn T4 as a user-initiated repair, the 
dialogue system has to be able to take into account both 
the deviation of the user from the expected behaviour, 
and the illocutionary force of her utterance (i.e. the de- 
nial of the system interpretation of T2). This deviation 
is the signal of an actual misunderstanding and it is 
related to the presence of a conversational implicature. 

The dialogue system initially interprets T4 with re- 
spect to its set of expectations. As soon as it realizes 
that the matched expectation realizes an implicature, 
it reinterprets T4 within the context of the utterance 
whose interpretation is now denied, i.e. the system 
traces back to the interpretation context of T2. The 



Figure 4: Example of misunderstanding 

In this example the sentence uttered in the turn T2 
was badly recognized. For the sake of simplicity, let us 
concentrate on what happened about the names of the 
departure and arrival city. Due to disturbances over the 
phone line, the uttered arrival city name ("Firenze") 
was not decoded. Moreover the phrase "from Pisa at 
eight" (in Italian, "da Pisa alle otto") was recognized as 
"Pisa Aeroporto". The system expectations were not 
satisfied since the name of the arrival city was not ac- 
quired. Then the dialogue system decided to trigger a 
confirmation plus initiative speech act in order to ob- 
tain the missing parameter and to confirm departure. 
That choice resulted in the generation of T3. At this di- 
alogue stage the set of expectations were the ones enu- 
merated in the previous paragraph, although applied 
to the arrival city parameter. In T4 the user offered an 
arrival city. This matched the first expectation of our 
list. At this stage neither the user nor the system had 
grasped the inconsistency concerning departure. Since 
it had not obtained explicit confirmation for departure, 
the dialogue system addressed the user with the Yes/No 
question of turn T5. The contextual interpretation of 
the user utterance T6 detects the explicit confirmation 
of the arrival city and the misunderstanding that had 
occurred during the recognition of the departure city. 
The latter is refocused again in T7, and the user is ad- 
dressed with a new Yes/No question. 



Architecture of DIALOGOS 

DIALOGOS is a real-time task-oriented system com- 
posed of the following modules: the acoustical front- 
end, the linguistic processor, the dialogue manager and 
the message generator, and the text-to-speech synthe- 
sizer. Its vocabulary size is about 3,500 words includ- 
ing 2,983 place names. The acoustical front-end and 
the synthesizer are connected to the telephone network 
through a telephone interface, while the dialogue man- 
ager is connected to a Computer Information System to 
obtain information about Italian Railway time-tables. 
The acoustical front-end performs feature extraction 
and acoustic-phonetic decoding. The recognition mod- 
ule is based on a frame synchronous Viterbi decoding, 
where the acoustic matching is performed by a phonetic 
neural network. During the recognition, it makes use 
of language models that are class-based bigrams trained 
on 30K sentences. The training data were partially de- 
rived from the trial of a previous spoken language sy s- 

tem appUnd tn iho samp domain ( Ragp-ia. nt al U 



T he linguistic processor starts from the best-decoded 
sequence, and performs a multi-step robust partial pars- 
ing. In this strategy, partial solutio ns are accepted ac- 
cording to the linguistic knowledge ( Baggia and RuUcnt 
199|). At the end of the parsing stage a deep seman- 
tic representation for the user utterance is sent to the 
dialogue module, that will be described in the next sub- 
section. 

The dialogue module performs contextual interpreta- 
tion and generates the answer wh ich is sent to the text - 
to-speech synthesizer Eloquens ( |Quazza et al. 1993 ), 
that contains specific prosodic rules for the Italian lan- 
guage. 

The dialogue system 

The dialogue strategy of DIALOGOS has been designed 
both to maintain the control of the cooperative interac- 
tion and to leave the expert user the freedom to guide 
the interaction. Refe rring to the classification reported 
in ( [Smith etal. 1995D , the DIALOGOS system is able to 
support directive, suggestive and declarative initiative 
modes. It only exploits the directive mode in order to 
deal with repetitive problems at the recognition and un- 
derstanding levels. The directive mode is implemented 
by automatic switching to the isolated word recognition 
modality. 

The flow of a typical interaction with the system may 
be divided into two main stages: the acquisition of a set 
of parameters, used to select a reasonably small set of 
objects of interest to the user, and the presentation of 
the information related to the retrieved objects. These 
two stages may lead to the combination of several subdi- 
alogues, each of them with its own purpose. The system 
is able to move properly through nested subdialogues 
thanks to the maintenance of a dialogue history which 
relates the focus of the current utterance to the appro- 
priate interpretation context. A detailed discussion of 



The system performs contextual interpretation on the 
basis of the model of user-system interaction discussed 
above. The model has been implemented using a Tran- 
sition Network formalism. Each node in the network 
represents a state in the dialogue: associated actions 
are executed when the interaction comes to that state. 
The actions are declared in a library of functions writ- 
ten in C. There are specific actions for each dialogue 
functionality, including contextual interpretation, up- 
dating of the context hierarchy (see below) , and gener- 
ation of dialogue predictions and expectations. A state 
transition is executed when the conditions associated 
with the arc are satisfied. Conditions are also library 
functions: they check both the current state of the dia- 
logue model and the content of the user's utterance, in 
order to direct the interaction into a new coherent state. 
Default transitions are associated with each node. 

The contextual interpretation of user utterances 
takes into account the li nguistic hist or y and the global 
and l ocal (active) focus (Grosz 1981), (3uri and McCoy 



1995). Each time a user utterance has to be interpreted. 



the dialogue strategy is given in (Gerbino and Danicli 
199|. 



all the information useful for its interpretation (specif- 
ically, its semantic content and some surface informa- 
tion that may be used to solve references) ar e stored in 
a cycle-structure (Baggia and Gerbino 1991). At each 
point of the dialogue, the linguistic history consists in 
the history of the previous cycle-structures. The inter- 
pretation of the utterance causes the creation of a local 
focus structure whi ch is linked to the cycle-st ructure 
that has caused it (Gerbino and Danieli 1993). The 
focus structures are hierarchically organized in a tree 
(that we name "context hierarchy"), whose root repre- 
sents the global focus at the beginning of the dialogue. 
A new node in the tree, that is a new active focus, is 
created and linked in the hierarchy when the user oper- 
ates a focus restriction or a focus shifting. The correct 
interpretation context of an utterance can be the active 
focus, if the utterance refers to the objects currently 
focused; when there is discrepancy between utterance 
focus and active focus, the hierarchy is climbed up for 
checking the semantic and pragmatic consistency be- 
tween the current utterance and the previous ones. The 
first node where the consistency is verified is chosen as 
the utterance interpretation context. 

Experimental Data 

Recently DIALOGOS has been tested in a large field 
trial. Five hundred subjects of different ages and lev- 
els of education called the system from all over Italy. 
They had never used a computerized telephone service 
before. Each of them made three calls to the system. 
The experimental material has been transcribed and it 
is currently being evaluated. 

Subjects called the system from their own places 
(80%), from public telephone booths (10%), and by mo- 
bile phones used in noise environment (such as street, 
underground stations, and so on). They received a sin- 
gle page of printed instructions which contained a brief 
explanation of the service capabilities. 



Each subject had to plan a trip from a city A to a 
city B in a certain date and time: they had to find out 
departure and arrival times of trains that satisfy their 
needs. The departure and the destination were specified 
in the scenarios, on the contrary the subjects had the 
freedom to choose the date and the time of the travel. 

Some features of the dialogues collected in the test 
and already evaluated are shown in Table |l|. The total 
number of evaluated dialogues, the number of continu- 
ous speech utterances, the number of isolated words, 
and the average number of continuous speech utter- 
ances per dialogue are reported. 



No. of 
Dialogs 


No. of 

Utterances 


No. of 
Iso. Words 


No. of 
Utt. per Dial. 


923 


9,124 


442 


9.8 



Table 1: Corpus 

A user-system interaction was considered successful 
when the user received all the information she needed. 
In particular, the transaction success is the measure of 
the success of the system in providing users with the 
train timetables they required. A definition of trans- 
action succe ss and related dialogue eva luation metrics 
is given in ( Danieli and Gerbino 1995 ). The rate of 
transaction success for the 923 evaluated dialogues is 
84%. We consider this result a promising starting point 
for further work within the conceptual framework de- 
scribed in this paper. 

Conclusions 

This paper discusses the use of expectations for deal- 
ing with miscommunication in spoken dialogue systems. 
It has been suggested that the deviation of the user's 
behaviour from predictions, along with the generation 
of a conversational implicature, are symptoms of actual 
miscommunication. The proposed analysis exploits two 
notions of expectations. The first one refers to predic- 
tions: they have been larger ly used by the speech com- 
munity to reduce speech and semantic interpretation 
errors. The second expectation notion gives insights on 
how to relate future user utterance to previous ones. 
The described approach has been implemented in the 
dialogue module of the DIALOGOS spoken language 
system. 

The level of appropriateness of the system's answer 
when miscommunication occurs is greatly enhanced 
when the spoken dialogue system is able to profit from 
both kinds of expectation. Some experimental data 
support this claim. Preliminary results (based on 923 
dialogues with naive users) show that the rate of trans- 
action success of the system is 84%. 

Although this analysis is only a first step towards 
an adequate handling of miscommunication within our 
automatic speech recognition system, it has been shown 
to be useful in improving the overall performance of the 
system. 
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